In many practical applications, the pose and illumination changes become the bottleneck for face recognition. Many existing works have been proposed to account for such variations. The pose-invariant methods may be generally separated into two categories: 2D-based and 3D-based. In the first category, poses are either handled by 2D image matching or by encoding a test image using some bases or exemplars. For example, in one conventional way, stereo matching is used to compute the similarity between two faces. a test face combination of training images is represented, and then the linear regression coefficients are utilized as features for face recognition. 3D-based methods usually capture 3D face data or estimate 3D models from 2D input, and try to match them to a 2D probe face image. Such methods make it possible to synthesize any view of the probe face, which makes them generally more robust to pose variation.
The illumination-invariant methods typically make assumptions about how illumination affects the face images, and use these assumptions to model and remove the illumination effect. For example, in the art, it has been designed a projector-based system to capture images of each subject in the gallery under a few illuminations, which can be linearly combined to generate images under arbitrary illuminations. With this augmented gallery, they adopted sparse coding to perform face recognition.
The above methods have certain limitations. For example, capturing 3D data requires additional cost and resources. Inferring 3D models from 2D data is an ill-posed problem. As the statistical illumination models are often summarized from controlled environment, they cannot be well generalized in practical applications.